This study will evaluate the effectiveness and safety of thalidomide for the treatment of oral and esophageal ulcers in patients with advanced HIV disease. It will also look at the effect of thalidomide on HIV load in HIV-infected individuals with aphthous ulcers. (ACTG 251)